Method for controlling head mounted display, and system for implemeting the method

ABSTRACT

A method for controlling an immersive head mounted display configured to provide a virtual space to a user. The method includes generating a virtual space image that forms a virtual space viewable by the user. The method further includes determining a reference line of sight. The method further includes determining a field-of-view region of the virtual space based on the reference line of sight. The method further includes generating a region of the virtual space image corresponding to the field-of-view region as a field-of-view image having an image quality higher than an image quality of a different portion of the virtual space image.

RELATED APPLICATIONS

The present application claims priority to Japanese Application Number2015-140224, filed Jul. 14, 2015, the disclosure of which is herebyincorporated by reference herein in its entirety.

TECHNICAL FIELD

This disclosure relates to a method for controlling a head mounteddisplay, and a system for implementing the method.

BACKGROUND ART

In Patent Literature 1, there is disclosed a head mounted display (HMD)which is wearable on a head of a user and configured to display aright-eye image and a left-eye image for a right eye and a left eye ofthe user, respectively, to thereby provide a three-dimensional virtualspace to the user. Further, in Patent Literature 2, it is disclosedthat, as a method of updating a three-dimensional virtual space image inreal time based on change in point of view of a user wearing an HMD, areal-time rendering method is employed.

[Patent Literature 1] Japanese Patent Application Laid-open No. Hei8-006708

[Patent Literature 2] Japanese Patent Application Laid-open No.2004-013326

SUMMARY

In order to provide a three-dimensional virtual space image, a right-eyeimage and a left-eye image are generated. Therefore, when thethree-dimensional virtual space image is provided in high image quality,a rendering load for generating each image is increased. The increase inrendering load becomes particularly noticeable when thethree-dimensional virtual space image is updated in real time based on achange in a point of view of the user. This disclosure helps to reducethe rendering load when the three-dimensional virtual space image isprovided.

According to this disclosure, there is provided a method for controllingan immersive head mounted display (hereinafter referred to as “HMD”)configured to provide a virtual space to a user. The method includesgenerating a virtual space image that forms a virtual space to which theuser is immersed. The method further includes determining a referenceline of sight. The method further includes determining a field-of-viewregion of the virtual space, which is visually recognized by the user,based on the reference line of sight. The method further includesgenerating a region of the virtual space image corresponding to thefield-of-view region as a field-of-view image having an image qualityhigher than an image quality of the virtual space image.

Further, according to this disclosure, there is provided a method forcontrolling an HMD configured to provide a virtual space to a user. Themethod includes generating a virtual space image that forms a virtualspace to which the user is immersed. The method further includesdetermining a reference line of sight. The method further includesdetermining a field-of-view region of the virtual space, which isvisually recognized by the user, based on the reference line of sight.The method further includes generating a region of the virtual spaceimage corresponding to the field-of-view region as a field-of-viewimage, in which, when a position of the user in the virtual spacearrives at an update position, the virtual space image is updated, and aregion of the updated virtual space image corresponding to thefield-of-view region is generated as the field-of-view image.

According to this disclosure, the rendering load when thethree-dimensional virtual space image is provided is reduced.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 A view of an HMD system according to at least one embodiment ofthis disclosure.

FIG. 2 An illustration of an orthogonal coordinate system on athree-dimensional space defined about a head of a user wearing an HMD.

FIG. 3A An XYZ space view for illustrating an example of acorrespondence relationship of arrangement positions of a virtual spaceand a real space.

FIG. 3B An XZ plane view for illustrating an example of thecorrespondence relationship of the arrangement positions of the virtualspace and the real space.

FIG. 4 A block diagram for illustrating a function of a control circuitunit, for achieving a function of the HMD system according to at leastone embodiment.

FIG. 5 A diagram for illustrating at least one example of a method ofdetermining a line-of-sight direction.

FIG. 6A A schematic three-dimensional view for illustrating afield-of-view region.

FIG. 6B A YZ plane view for illustrating the field-of-view region asviewed from an X direction.

FIG. 6C An XZ plane view for illustrating the field-of-view region asviewed from a Y direction.

FIG. 7 A diagram for illustrating a field-of-view image according to atleast one embodiment.

FIG. 8 A flow chart for illustrating processing for achieving thefunction of the HMD system according to at least one embodiment.

FIG. 9 A flow chart for illustrating processing for achieving thefunction of the HMD system according to at least one embodiment.

FIG. 10 A diagram for illustrating a field-of-view image according to atleast one embodiment.

FIG. 11 A diagram for illustrating a field-of-view image according to atleast one embodiment.

FIG. 12 A diagram for illustrating a field-of-view image according to atleast one embodiment.

FIG. 13 A diagram for illustrating at least one example of thefield-of-view image when a game stage is updated.

DETAILED DESCRIPTION

First, at least one embodiment of this disclosure is described byenumerating contents thereof. A method for controlling an HMD and asystem for implementing the method according to at least one embodimentof this disclosure include the following configurations.

(Item 1)

A method for controlling an HMD configured to provide a virtual space toa user. The method includes generating a virtual space image that formsa virtual space to which the user is immersed. The method furtherincludes determining a reference line of sight. The method furtherincludes determining a field-of-view region of the virtual space, whichis visually recognized by the user, based on the reference line ofsight. The method further includes generating a region of the virtualspace image corresponding to the field-of-view region as a field-of-viewimage having an image quality higher than an image quality of thevirtual space image.

According to the method for controlling an HMD of this item, the virtualspace image is generated in advance in a low image quality, and then thepart of the virtual space image corresponding to the user'sfield-of-view region is generated as a field-of-view image having a highimage quality. Thus, rendering resources can be concentrated on the partof the virtual space image corresponding to the user's field-of-viewregion. With this, the rendering load when the three-dimensional virtualspace image is provided may be reduced.

(Item 2)

A method for controlling an HMD according to Item 1, further includingdetecting a direction in which the reference line of sight moves.

When the field-of-view region moves from a first part to a second partof the virtual space image due to movement of the reference line ofsight, the field-of-view image is generated in the second part beforecompletion of movement of the field-of-view region.

According to the method for controlling an HMD of this item, beforecompletion of the movement of the field-of-view region, thefield-of-view image of the region is generated. Thus, the user mayalways visually recognize a virtual space having a high image quality.With this, the user experience may be improved.

(Item 3)

A method for controlling an HMD according to Item 1, in which thefield-of-view image is generated so as to cover a predetermined regionaround the field-of-view region.

According to the method for controlling an HMD of this item, thefield-of-view image covers the field-of-view region. Thus, generation ofthe field-of-view image of the field-of-view region before completion ofthe movement of the field-of-view region is facilitated. With this, theuser experience may be improved.

(Item 4)

A method for controlling an HMD according to Item 3, further including astep of detecting a direction in which the reference line of sightmoves.

The predetermined region is set so as to be increased toward thedirection in which the reference line of sight moves.

According to the method for controlling an HMD of this item, a region inwhich the field-of-view image is generated outside of the field-of-viewregion is set large in the direction in which the reference line ofsight moves. Thus, generation of the field-of-view image of thefield-of-view region before completion of the movement of thefield-of-view region is facilitated.

(Item 5)

A method for controlling an HMD according to any one of Items 1 to 4,

when a position of the user in the virtual space arrives at an updateposition, the virtual space image is updated.

A region of the updated virtual space image corresponding to thefield-of-view region is generated as a field-of-view image having animage quality higher than the image quality of the virtual space image.

According to the method for controlling an HMD of this item, when theposition of the user in the virtual space arrives at the updateposition, the virtual space image is collectively updated. With this,the virtual space image may be generated efficiently, and the renderingload may be reduced.

(Item 6)

A method for controlling an HMD configured to provide a virtual space toa user. The method includes generating a virtual space image that formsa virtual space to which the user is immersed. The method furtherincludes determining a reference line of sight. The method furtherincludes determining a field-of-view region of the virtual space, whichis visually recognized by the user, based on the reference line ofsight. The method further includes generating a region of the virtualspace image corresponding to the field-of-view region as a field-of-viewimage. When a position of the user in the virtual space arrives at anupdate position, the virtual space image is updated, and a region of theupdated virtual space image corresponding to the field-of-view region isgenerated as the field-of-view image.

According to the method for controlling an HMD of this item, when theposition of the user in the virtual space arrives at the updateposition, the virtual space image is collectively updated. With this,the virtual space image may be generated efficiently, and the renderingload may be reduced.

(Item 7)

A program for controlling an HMD, which causes a computer to execute themethod for controlling an HMD of any one of Items 1 to 6.

With this, an intuitive operational feeling can be provided to the user.

Specific examples of a method for controlling an HMD and a program forcontrolling an HMD according to at least one embodiment of thisdisclosure are described below with reference to the drawings. Thisdisclosure is not limited to those examples, and is defined by Scope ofClaims. This disclosure includes modifications within Scope of Claimsand the equivalents thereof. In the following description, like elementsare denoted by like reference symbols in the description of thedrawings, and redundant description thereof is omitted.

FIG. 1 is an illustration of an HMD system 100 including an HMD 110according to at least one embodiment. The HMD system 100 includes theHMD 110 to be worn on a head of a user, a control circuit unit 120, aninclination sensor 130, and an eye gaze sensor 140.

The HMD 110 includes a display 112 that is a non-transmissive displaydevice, a sensor unit 114, and the eye gaze sensor 140. In at least oneembodiment, display 112 is partially transmissive. The control circuitunit 120 is configured to cause the display 112 to display a right-eyeimage and a left-eye image, to thereby provide a three-dimensional imageusing binocular parallax as a virtual space. The display 112 is arrangedright in front of the user's eyes, and thus the user can be immersed tothe virtual space. The virtual space includes a background, variousobjects that can be operated by the user, menu images, and the like.

The display 112 may include a right-eye sub-display configured toprovide a right-eye image, and a left-eye sub-display configured toprovide a left-eye image. Further, as long as the right-eye image andthe left-eye image can be provided, the display 112 may be constructedof one display device. For example, a shutter configured to enablerecognition of a display image with only one eye may be switched at highspeed, to thereby independently provide the right-eye image and theleft-eye image.

The eye gaze sensor 140 has an eye tracking function of detectingline-of-sight directions of the user's right and left eyes. In at leastone embodiment, the eye gaze sensor 140 includes a right-eye sensor anda left-eye sensor, which are respectively configured to detect theline-of-sight directions of the right and left eyes, to thereby detect adirection in which the user focuses his/her gaze. The eye gaze sensor140 can employ a known sensor having an eye tracking function. Forexample, infrared light may be radiated to each of the right eye and theleft eye to acquire reflection light from the cornea or the iris, tothereby obtain a rotational angle of the eyeball.

The control circuit unit 120 is a computer to be connected to the HMD110, and is configured to provide the virtual space to the display 112,to thereby execute processing so as to operate various objects displayedin the virtual space or display and control various menu images and thelike. The control circuit unit 120 stores a program for controllingexecution of such operations. The control circuit unit 120 is notrequired to be mounted on the HMD 110, and may be constructed asseparate hardware (for example, a known personal computer, or a servercomputer via a network). Further, in at least one embodiment, only apart of the functions of the control circuit unit 120 is mounted on theHMD 110, and the remaining functions thereof are mounted on differenthardware.

The inclination sensor 130 is configured to detect information relatingto a position and movement of the HMD 110. The inclination sensor 130includes the sensor unit 114 and a detection unit 132. The sensor unit114 may include a plurality of light sources. The light source is, forexample, an LED configured to emit an infrared ray. The detection unit132 is, for example, an infrared sensor, and is configured to detect theinfrared ray from the light source as a detection point of the HMD 110,to thereby detect over time information relating to an angle in a realspace of the HMD 110 based on the movement of the user. Then, the timechange of the angle of the HMD 110 can be detected based on the temporalchange of the information detected by the detection unit 132, and thusinformation relating to the position and the movement of the HMD 110 canbe detected.

The information relating to the angle acquired by the inclination sensor130 is described with reference to FIG. 2. The XYZ axes are definedabout the head of the user wearing the HMD 110. A perpendiculardirection in which the user stands upright is defined as the Y axis, afront-rear direction being orthogonal to the Y axis and connectingbetween the user and the center of the display 112 is defined as the Zaxis, and a lateral direction orthogonal to the Y axis and the Z axis isdefined as the X axis. Then, inclination angles θx (so-called pitchangle), θy (so-called yaw angle), and θz (so-called roll angle) of theHMD 110 about the respective axes are detected. Thus, the informationrelating to the position and the movement of the HMD 110 can be detectedbased on the temporal change of those angles.

In at least one embodiment, the inclination sensor 130 is constructed ofonly one of the detection unit 132 and the sensor unit 114 fixed nearthe display 112. In at least one embodiment, the sensor unit 114includes a geomagnetic sensor, an acceleration sensor, or an angularvelocity sensor (gyroscope), and is configured to use at least one ofthose sensors to detect the inclination of the HMD 110 (in particular,the display 112) worn on the head of the user. With this, theinformation relating to the position and the movement of the HMD 110 canbe detected. For example, the angular velocity sensor can detect overtime the angular velocity about three axes of the HMD 110 based on themovement of the HMD 110, and can determine the time change of the angle(inclination) about each axis. In this case, the detection unit 132 maybe omitted. Further, the detection unit 132 may include an opticalcamera. In this case, the information relating to the position and themovement of the HMD 110 can be detected based on the image information,and thus the sensor unit 114 is omitted, in at least one embodiment.

A function of detecting the information relating to the position and themovement of the HMD 110 with use of the inclination sensor 130 isreferred to as “position tracking”. The relationship between theposition tracking performed by the inclination sensor 130 and a virtualcamera 1 arranged in a virtual space 2 is described with reference toFIG. 3A and FIG. 3B. In order to describe the positional relationshipbetween the virtual camera 1 and the sensor 130, in the following, theposition of the sensor 130 is set as a position of the detection unit132 when the detection unit 132 is provided, and is set as the positionof the sensor unit 114 when the detection unit 132 is not provided. FIG.3A is a schematic three-dimensional view for illustrating therelationship between the virtual space 2 and the inclination sensor 130in the real space, and FIG. 3B is a plan view for illustrating therelationship between the virtual space 2 and the sensor 130 in the realspace as viewed from the Y direction. The virtual camera 1 is arrangedinside the virtual space 2, and the inclination sensor 130 is virtuallyarranged outside of the virtual space 2 (in the real space).

The virtual space 2 is formed into a celestial sphere shape having aplurality of substantially-square or substantially-rectangular meshsections 3. Each mesh section is associated with space information ofthe virtual space 2, and, as described later, a field-of-view image isformed and a field-of-view region is defined based on this spaceinformation. In at least one embodiment, as illustrated in FIG. 3B, inan XZ plane, a center point 21 of the celestial sphere is adjusted toalways be arranged on a line connecting between the virtual camera 1 andthe sensor 130. For example, when the user wearing the HMD moves, andthus the position of the virtual camera 1 moves in the X direction, theregion of the virtual space 2 is changed such that the center 21 ispositioned on the line segment between the virtual camera 1 and theinclination sensor 130.

The HMD system 100 may include headphones including a microphone in anyof the elements. With this, the user can give audible instructions to apredetermined object in the virtual space. Further, the HMD system 100may include a television receiver in any of the elements in order toreceive a broadcast of a television program on a virtual television inthe virtual space. Further, as described later, the HMD system 100 mayhave a communication function or the like in order to display anelectronic mail or the like that the user has acquired.

FIG. 4 is a diagram for illustrating the function of the control circuitunit 120 for achieving display processing of the virtual space 2 in theHMD system 100 and operations of various menu displays and objects to bedisplayed in the virtual space 2. The control circuit unit 120 isconfigured to control an image to be output to the display 112 based onthe input from the inclination sensor 130 and the eye gaze sensor 140.

The control circuit unit 120 includes a display control unit 200 and anobject control unit 300. The display control unit 200 includes a virtualspace image generating unit 210, an HMD movement detecting unit 220, aline-of-sight direction detecting unit 230, a point-of-gaze specifyingunit 240, a field-of-view region determining unit 250, a field-of-viewimage generating unit 260, and a space information storing unit 270. Theobject control unit 300 includes an object information storing unit 310and a virtual camera information storing unit 320.

The inclination sensor 130 and the eye gaze sensor 140 are eachconnected to the display control unit 200 and the object control unit300 so as to enable communication therebetween, and may be connected viaa wired or wireless communication interface. The display control unit200 and the object control unit 300 are connected to the display 112 soas to enable communication therebetween, and may be connected via awired or wireless communication interface. The space information storingunit 270, the object information storing unit 310, and the virtualcamera information storing unit 320 include various types of data forproviding, to the display 112, output information corresponding to theinput from the inclination sensor 130 and the eye gaze sensor 140.

The inclination sensor 130 is configured to output, to the displaycontrol unit 200 and the object control unit 300, the informationrelating to the position and the movement of the HMD 110 based on theangle information that is detected over time by the sensor unit 114. Theinformation relating the position and movement of the HMD 1100 is alsobased on the angle information of the HMD 110, which is detected overtime by the detection unit 132.

The virtual space image generating unit 210 is configured to read, fromthe space information storing unit 270, information relating to thevirtual space 2 to which the user is immersed, to thereby generate avirtual space image. The virtual space image is a 360-degree panoramicimage for forming a predetermined game space independent of aline-of-sight direction of the user. The virtual space image is formedto have an image quality lower than that of the field-of-view image thatis within a determined field of view of the user. The virtual spaceimage may have, for example, a rendering level of about 30% of that ofthe field-of-view image, and may be achieved by, for example, reducing apolygon count for forming the field-of-view image.

The HMD movement detecting unit 220 is configured to detect the movementof the HMD 110 based on the input information from the inclinationsensor 130. Further, the HMD movement detecting unit 220 is configuredto detect a field-of-view direction for defining the direction of theHMD 110 in the virtual space 2. Further, the HMD movement detecting unit220 is configured to output the detected field-of-view direction to thefield-of-view region determining unit 250.

The line-of-sight direction detecting unit 230 is configured to detectdirections of the lines of sight of the user's right and left eyes basedon the input information from the eye gaze sensor 140. The point-of-gazespecifying unit 240 is configured to specify the point of gaze at whichthe user focuses his/her gaze based on the user's line-of-sightinformation from the eye gaze sensor 140.

As illustrated in FIG. 5, the eye gaze sensor 140 detects the directionsof the lines of sight of the right and left eyes of a user U. When theuser U is looking at a near place, lines of sight R1 and L1 aredetected, and a point of gaze N1 being an intersection of the lines ofsight R1 and L1 is specified. Further, when the user is looking at a farplace, lines of sight R2 and L2, which form smaller angles with the Zdirection as compared to the lines of sight R1 and L1, are specified.After the point of gaze N1 is specified, a line-of-sight direction NO ofthe user U is specified. The line-of-sight direction NO is a directionin which the line of sight of the user U is actually directed with botheyes. The line-of-sight direction NO is defined as, for example, anextension direction of a straight line that passes through the point ofgaze N1 and the middle of a right eye R and a left eye L of the user U.

The field-of-view region determining unit 250 is configured to determinethe field-of-view region of the virtual camera 1 in the virtual space 2based on the virtual space information stored in the space informationstoring unit 270 and on the input information from the inclinationsensor 130 and the eye gaze sensor 140. The field-of-view imagegenerating unit 260 is configured to generate, as a field-of-view image,a part of the 360-degree panoramic image forming the virtual space,based on the information relating to the field-of-view region. Thefield-of-view image is output to the display 112. The field-of-viewimage includes two two-dimensional images for the left eye and the righteye, and those images are superimposed on the display 112, to therebyprovide the virtual space 2 being a three-dimensional image to the user.

Referring to FIG. 6A to FIG. 6C, a field-of-view region 23, which isdetermined by the field-of-view region determining unit 250 along thecelestial sphere surface of the virtual space 2, is described. FIG. 6Ais a schematic three-dimensional view for illustrating the field-of-viewregion 23. FIG. 6B is a YZ plane view of the field-of-view region 23 asviewed from the X direction. FIG. 6C is an XZ plane view of thefield-of-view region 23 as viewed from the Y direction. As illustratedin FIG. 6A, the field-of-view region 23 forms a part of a virtual spaceimage 22. The field-of-view region 23 is, as described later, a part ofthe virtual space image 22 forming the field of view of the user. Thefield-of-view region 23 is determined based on a reference line of sight5, and the reference line of sight 5 is determined based on the positionand the direction of the virtual camera 1. In at least one embodiment,the reference line of sight 5 is defined based on at least one of thefield-of-view direction defining the direction of the HMD 110, or theline-of-sight direction NO specified by the eye gaze sensor 140.

The field-of-view region 23 has a first region 24 (see FIG. 6B) that isa range defined by the reference line of sight 5 and a YZ cross sectionof the virtual space image 22, and a second region 25 (see FIG. 6C) thatis a range defined by the reference line of sight 5 and an XZ crosssection of the virtual space image 22. The first region 24 is set as arange including a polar angle α with the reference line of sight 5 beingthe center. The second region 25 is set as a range including an azimuthβ with the reference line of sight 5 being the center.

The field-of-view image generating unit 260 is configured to generate aregion of the virtual space image 22 formed in advance corresponding tothe field-of-view region 23 as a field-of-view image 26 having an imagequality higher than that of the virtual space image 22. Thefield-of-view image 26 is an image forming a space that is actuallyvisually recognized by the user in the virtual space 2. FIG. 7 is adiagram for illustrating a state in which the field-of-view image 26 isgenerated based on the field-of-view region 23. In at least oneembodiment, the field-of-view image 26 is generated so as to cover apredetermined region around the field-of-view region 23. Further, thefield-of-view image 26 is generated as an image having an image qualityhigher than that of the virtual space image 22, in at least oneembodiment. Specifically, the higher image quality may be obtained asfollows. As illustrated in FIG. 7, in at least one embodiment, thepolygon count of the image forming the field-of-view image 26 isincreased to be higher than that of the image forming the virtual spaceimage 22, to thereby increase the rendering level. Otherwise, in atleast one embodiment, the virtual space image 22 is subjected to texturemapping.

The object control unit 300 is configured to specify an object to beoperated based on information on the object in the virtual space, whichis stored in the object information storing unit 310, and on the userinstruction from the inclination sensor 130 and the eye gaze sensor 140.Then, the virtual camera information stored in the virtual camerainformation storing unit 320 is adjusted based on a predetermined useroperation instruction to the object to be operated. The adjusted virtualcamera information is output to the display control unit 200, and thusthe field-of-view image is adjusted. Further, an operation correspondingto the predetermined user operation instruction is executed to theobject to be operated, and the object control information is output tothe display 112 and the display control unit 200. Specific processing ofthe object operation is described later.

A hardware element for achieving each function of the control circuitunit 120 can be constructed of a CPU, a memory, and other integratedcircuits. Further, each function is achieved by various programs servingas software elements loaded in the memory. Therefore, a person skilledin the art would understand that those functional blocks can be achievedby hardware, software, or a combination thereof.

Referring to FIG. 8 and FIG. 9, description is given of a processingflow of the HMD system 100, for generating the field-of-view image 26based on the virtual space image 22 and the reference line of sight 5.The field-of-view image generation processing may be achieved by theinteraction between the HMD 110 (eye gaze sensor 140, inclination sensor130) and the control circuit unit 120.

The control circuit unit 120 (virtual space image generating unit 210)generates the virtual space image 22 in order to provide the virtualspace 2 to which the user is immersed (S120-1). When an operation, e.g.,movement or inclination, is input to the HMD from the user (S110-1), theinclination sensor 130 detects the position and the inclination of theHMD 110 (S130-1). The detection information of the inclination sensor130 is transmitted to the control circuit unit 120, and the HMD movementdetecting unit 220 determines the position information and theinclination information of the HMD 110. With this, the field-of-viewdirection is determined based on the position information and theinclination information of the HMD 110 (S120-2).

When the eye gaze sensor 140 detects the movement of the eyeballs of theuser's right and left eyes (S140-1), the information is transmitted tothe control circuit unit 120. When the line-of-sight direction detectingunit 230 of the control circuit unit 120 specifies the lines of sight ofthe right and left eyes, the point-of-gaze specifying unit 240 specifiesthe user's point of gaze, to thereby specify the line-of-sightdirection.

The field-of-view region determining unit 250 specifies the referenceline of sight 5 based on the field-of-view direction or theline-of-sight direction (S120-4). The field-of-view region determiningunit 250 determines the field-of-view region 23 (first region 24 andsecond region 25) based on the reference line of sight 5 (S120-5). Thefield-of-view image generating unit 260 generates the field-of-viewimage 26 based on the field-of-view region 23 (S120-6). As describedabove, the field-of-view image 26 is generated by increasing the imagequality of a region of the virtual space image 22 corresponding to thefield-of-view region 23. The HMD 110 receives the information relatingto the field-of-view image 26 from the control circuit unit 120, andcauses the display 112 to display the field-of-view image 26 (S110-2). Astate in which the field-of-view image 26 is generated in a part of thevirtual space image 22 as described above is illustrated in FIG. 10.

As illustrated in FIG. 9, when the user inputs an operation of movingthe reference line of sight 5 to the HMD 110 or the eye gaze sensor 140,the control circuit unit 120 detects the movement of the reference lineof sight 5 (S120-7), and updates the field-of-view image 26 (S120-8).With this, the user can visually recognize the updated field-of-viewimage 26, and can obtain a sense of immersion in the virtual space 2.

At this time, the field-of-view image 26 moves from a first part 27illustrated in FIG. 10 to a second part 28 illustrated in FIG. 11. Thevirtual space image 22 in the second part 28 is generated as thefield-of-view image 26 due to the movement of the reference line ofsight 5 by the user described above. In at least one embodiment, thevirtual space image 22 in the second part 28 is generated as thefield-of-view image 26 before completion of the movement from the firstpart to the second part of the field-of-view region 23 due to themovement of the reference line of sight 5. With this, the user canalways visually recognize the virtual space 2 having a high imagequality, and the user experience can be improved.

The field-of-view image 26 is generated so as to cover a predeterminedregion around the field-of-view region 23. In at least one embodiment,as illustrated in FIG. 12, the predetermined region is set so as to beincreased toward the direction in which the reference line of sight 5moves. In the direction in which the reference line of sight 5 moves,the field-of-view image 26 having a high image quality is generated inadvance in a wide range. Thus, before completion of the movement of thefield-of-view region 23, the field-of-view image 26 of the region can beeasily generated.

Further, when the HMD 110 detects the movement of the position of theuser in the virtual space 2, whether or not the user has arrived at anupdate position is determined (S120-9). FIG. 13 is an illustration of anexample of the field-of-view image 26 including an update position 29.The user can operate the HMD 110 and the external controller, to therebyoperate an object (game character) O, in at least one embodiment. Thefield-of-view image 26 in FIG. 13 is a part of the virtual space image22 forming a predetermined game stage. The user operates the object O tocause the object O to arrive at the update position 29, and thus thegame stage can proceed to the next stage.

After the game stage proceeds to the next stage, the virtual space image22 is updated to a virtual space image 22 corresponding to the gamestage (S120-10). The virtual space image 22 is generated as an imagehaving an image quality lower than that of the field-of-view image 26that is visually recognized by the user. Next, similarly to theabove-mentioned processing flow, the field-of-view region 23 isdetermined based on the reference line of sight 5 of the user (S120-11),and a region of the virtual space image 22 corresponding to thefield-of-view region 23 is generated as the field-of-view image 26having a high image quality (S120-12). Thus, the update of the gamestage is completed. In this process, when the position of the user inthe virtual space 2 arrives at the update position 29, the virtual spaceimage 22 is collectively updated. With this, the virtual space image 22can be generated efficiently, and the rendering load can be reduced.

The embodiment of this disclosure is described above. However, thisdisclosure is not limited to the embodiments described above. A personskilled in the art would understand that various modifications may bemade to the embodiments as long as the modifications do not deviate fromthe spirit and scope of this disclosure described in the claimsdescribed above.

The invention claimed is:
 1. A method for controlling an immersive headmounted display (HMD) configured to provide a virtual space to a user,the method comprising: generating and storing virtual space image datahaving a first image quality, wherein the virtual space image data isusable for forming a virtual space viewable by the user; generating avirtual space image, wherein generating the virtual space imagecomprises reading the previously stored virtual space image data;determining a reference line of sight; determining a field-of-viewregion of the virtual space based on the reference line of sight; andgenerating, using a computer, in the field-of-view region, afield-of-view image having a second image quality higher than the firstimage quality, wherein generating the virtual space image furthercomprises reducing a 3D polygon count as compared to the field-of-viewimage, and wherein generating the field-of-view image comprisesincreasing the 3D polygon count of the virtual space image, wherein thefield-of-view image corresponds in size to the field-of-view region,wherein the virtual space image outside of the field-of-view region isnot visible to the user, and wherein the virtual space image isgenerated outside the field-of-view region simultaneously with thegenerating of the field-of-view image.
 2. The method for controlling theHMD according to claim 1, further comprising detecting a direction inwhich the reference line of sight moves, wherein, when the field-of-viewregion moves from a first part to a second part of the virtual spaceimage due to movement of the reference line of sight, the field-of-viewimage is generated in the second part prior to completion of movement ofthe field-of-view region.
 3. The method for controlling the HMDaccording to claim 1, wherein the field-of-view image is generated so asto cover a predetermined region around the field-of-view region.
 4. Themethod for controlling the HMD according to claim 3, further comprisingdetecting a direction in which the reference line of sight moves,wherein the predetermined region is set to increase in the direction inwhich the reference line of sight moves.
 5. The method for controllingthe HMD according to claim 1, wherein the virtual space image is a360-degree panoramic image forming a predetermined game spaceindependent of the line-of-sight.
 6. A method for controlling animmersive head mounted display (HMD) configured to provide a virtualspace to a user, the method comprising: generating and storing virtualspace image data having a first image quality; generating a virtualspace image, based on the virtual space image data; determining areference line of sight; determining a field-of-view region of thevirtual space based on the reference line of sight; and generating,using a computer, in the field-of-view region, a field-of-view imagehaving a second image quality higher than the first image quality,wherein the field-of-view image is generated by subjecting the virtualspace image to texture mapping, wherein the virtual space image is a360-degree panoramic image forming a predetermined game spaceindependent of the line-of-sight, the virtual space image outside of thefield-of-view region is not visible to the user, the field-of-view imagecorresponds in size to the field-of-view region, and the virtual spaceimage is generated outside the field-of-view region simultaneously withthe generating of the field-of-view image.
 7. The method for controllingthe HMD according to claim 6, further comprising detecting a directionin which the reference line of sight moves, wherein, when thefield-of-view region moves from a first part to a second part of thevirtual space image due to movement of the reference line of sight, thefield-of-view image is generated in the second part prior to completionof movement of the field-of-view region.
 8. The method for controllingthe HMD according to claim 6, wherein the field-of-view image isgenerated so as to cover a predetermined region around the field-of-viewregion.
 9. The method for controlling the HMD according to claim 8,further comprising detecting a direction in which the reference line ofsight moves, wherein the predetermined region is set to increase in thedirection in which the reference line of sight moves.
 10. A system forcontrolling an immersive head mounted (HMD), wherein the systemcomprises: memory configured to store instructions; and a computerconnected to the memory, wherein the computer is configured to executethe instructions for: generating and storing virtual space image data,corresponding to a virtual space image, which has a first image qualityand forms a virtual space viewable by the user; determining a referenceline of sight; determining a field-of-view region of the virtual spacebased on the reference line of sight; and generating, in thefield-of-view region, a field-of-view image having a second imagequality higher than the first image quality, wherein the field-of-viewimage is generated by increasing a 3D polygon count of the virtual spaceimage, wherein the field-of-view image corresponds in size to thefield-of-view region, wherein the virtual space image outside of thefield-of-view region is not visible to the user, and wherein the virtualspace image is generated outside the field-of-view region simultaneouslywith the generating of the field-of-view image.
 11. The method forcontrolling the HMD according to claim 10, wherein the virtual spaceimage is a 360-degree panoramic image forming a predetermined game spaceindependent of the line-of-sight.